Skip to content

Conversation

roger-zhangg
Copy link
Member

Problem

3 flaky appbuilder tests

  2 failing
  1) AppBuilder Walkthrough
       Create project
         start with an existing template
           override existing template
             download serverlessland proj:

      AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
+ actual - expected

+ 'Encountered an unexpected error when processing the request, please try again'
- 'template.yaml already exist in the selected directory, overwrite?'
      + expected - actual

     Error: Test length exceeded max duration: 30 seconds
[No Pending UI Elements Found]

      -Encountered an unexpected error when processing the request, please try again
      +template.yaml already exist in the selected directory, overwrite?
      
      at /Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/test/awsService/appBuilder/walkthrough.test.ts:196:32
      at s.value (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/test/shared/vscode/window.ts:502:35)
      at Proxy.z (/private/tmp/.vscode-test/vscode-darwin-arm64-1.83.0/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:80:1902)
      at Proxy.fire (/private/tmp/.vscode-test/vscode-darwin-arm64-1.83.0/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:80:2119)
      at fireOnDidShowMessage (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/test/shared/vscode/window.ts:147:35)
      at /Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/test/shared/vscode/message.ts:247:27
      at new Promise (<anonymous>)
      at Proxy.<anonymous> (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/test/shared/vscode/message.ts:244:20)
      at errorPromptHelper (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/codewhisperer/commands/startSecurityScan.ts:485:28)
      at startSecurityScan (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/codewhisperer/commands/startSecurityScan.ts:363:13)

  2) SamDebugConfigurationProvider
       makeConfig
         failure modes:
     AssertionError [ERR_ASSERTION]: Missing expected rejection.
      at async Context.<anonymous> (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/test/shared/sam/debugger/samDebugConfigProvider.test.ts:320:13)

  1) SAM runBuild
       :) path
         [entry: command palette] use samconfig should instantiate correct process in terminal:

      AssertionError [ERR_ASSERTION]: RunInterminal

1 !== 2

      + expected - actual

      -1
      +2
      
      at verifyCorrectDependencyCall (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/test/shared/sam/build.test.ts:344:20)
      at Context.<anonymous> (/Users/runner/work/aws-toolkit-vscode/aws-toolkit-vscode/packages/core/src/test/shared/sam/build.test.ts:532:13)

https://d1ihu6zq92vp9p.cloudfront.net/b4a5497f-734f-491d-a40e-795359615103/report.html

https://d1ihu6zq92vp9p.cloudfront.net/2d3ec18e-1584-4a40-948c-18babdfee24e/report.html
3 integ test constantly failing, 2 due to node 18 deprecated, 1 due to an uncaught change before

  1) SAM Integration Tests
       SAM runtime: nodejs18.x (ZIP)
         Starting with a newly created nodejs18.x (ZIP) SAM Application...
           "before all" hook for "produces an Add Debug Configuration codelens":
     CodeExpectedError: cannot open file:///codebuild/output/src2664791430/src/github.com/aws/aws-toolkit-vscode/packages/core/dist/src/testFixtures/workspaceFolder/inttestco5XDH/nodejs18.x/samapp-WKxmw3/testProject/hello-world/app.mjs. Detail: Unable to read file '/codebuild/output/src2664791430/src/github.com/aws/aws-toolkit-vscode/packages/core/dist/src/testFixtures/workspaceFolder/inttestco5XDH/nodejs18.x/samapp-WKxmw3/testProject/hello-world/app.mjs' (Error: Unable to resolve nonexistent file '/codebuild/output/src2664791430/src/github.com/aws/aws-toolkit-vscode/packages/core/dist/src/testFixtures/workspaceFolder/inttestco5XDH/nodejs18.x/samapp-WKxmw3/testProject/hello-world/app.mjs')
      at s.$tryOpenDocument (vscode-file://vscode-app/tmp/.vscode-test/vscode-linux-x64-1.83.0/resources/app/out/vs/workbench/workbench.desktop.main.js:2343:37719)

  2) SAM Integration Tests
       SAM runtime: nodejs18.x (Image)
         Starting with a newly created nodejs18.x (Image) SAM Application...
           "before all" hook for "produces an Add Debug Configuration codelens":
     SamCliError: Please verify your location. The following types of location are supported: [SamCliFailed]
  
      at makeUnexpectedExitCodeError (/codebuild/output/src2664791430/src/github.com/aws/aws-toolkit-vscode/packages/core/src/shared/sam/cli/samCliInvokerUtils.ts:48:12)
      at logAndThrowIfUnexpectedExitCode (/codebuild/output/src2664791430/src/github.com/aws/aws-toolkit-vscode/packages/core/src/shared/sam/cli/samCliInvokerUtils.ts:67:11)
      at runSamCliInit (/codebuild/output/src2664791430/src/github.com/aws/aws-toolkit-vscode/packages/core/src/shared/sam/cli/samCliInit.ts:65:36)
      at async createSamApplication (/codebuild/output/src2664791430/src/github.com/aws/aws-toolkit-vscode/packages/core/src/testInteg/sam.test.ts:640:9)
      at async Context.<anonymous> (/codebuild/output/src2664791430/src/github.com/aws/aws-toolkit-vscode/packages/core/src/testInteg/sam.test.ts:382:21)

  3) Application Builder
       application nodes in workspace (Test in order)
         3: contains correct resource node properties:

      AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:

1 !== 0

      + expected - actual

      -1
      +0
      
      at Context.<anonymous> (/codebuild/output/src2664791430/src/github.com/aws/aws-toolkit-vscode/packages/core/src/testInteg/appBuilder/sidebar/appBuilderNode.test.ts:132:20)
      at processTicksAndRejections (node:internal/process/task_queues:95:5)

Solution

  • make sure codewhisper doesn't show error in unrelated test
  • ensure clean state in sam tests
  • remove deprecated node 18 test cases

  • Treat all work as PUBLIC. Private feature/x branches will not be squash-merged at release time.
  • Your code changes must meet the guidelines in CONTRIBUTING.md.
  • License: I confirm that my contribution is made under the terms of the Apache 2.0 license.

@roger-zhangg roger-zhangg requested a review from a team as a code owner October 3, 2025 22:27
Copy link

⏳ I'm reviewing this pull request for security vulnerabilities and code quality issues. I'll provide an update when I'm done

Copy link

✅ I finished the code review, and didn't find any security or code quality issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants